Time-Contrastive Learning Based Unsupervised DNN Feature Extraction for Speaker Verification

In this paper, we present a time-contrastive learning (TCL) based unsupervised bottleneck (BN) feature extraction method for speech signals with an application to speaker verification. The method exploits the temporal structure of a speech signal and more specifically, it trains deep neural networks (DNNs) to discriminate temporal events obtained by uniformly segmenting the signal without using any label information, in contrast to conventional DNN based BN feature extraction methods that train DNNs using labeled data to discriminate speakers or passphrases or phones or a combination of them. We consider different strategies for TCL and its combination with transfer learning. Experimental results on the RSR2015 database show that the TCL method is superior to the conventional speaker and pass-phrase discriminant BN feature and Mel-frequency cepstral coefficients (MFCCs) feature for text-dependent speaker verification. The unsupervised TCL method further has the advantage of being able to leverage the huge amount of unlabeled data that are often available in real life.